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Abstract —We consider network aggregative games to model 
and study multi-agent populations in which each rational agent is 
influenced by the aggregate behavior of its neighbors, as specified 
by an underlying network. Specifically, we examine systems 
where each agent minimizes a quadratic cost function, that 
depends on its own strategy and on a convex combination of the 
strategies of its neighbors, and is subject to personalized convex 
constraints. We analyze the best response dynamics and we 
propose alternative distributed algorithms to steer the strategies 
of the rational agents to a Nash equilibrium configuration. The 
convergence of these schemes is guaranteed under different 
sufficient conditions, depending on the matrices defining the cost 
and on the network. Additionally, we propose an extension to the 
network aggregative game setting that allows for multiple rounds 
of communications among the agents, and we illustrate how it 
can be combined with consensus theory to recover a solution 
to the mean field control problem in a distributed fashion, 
that is, without requiring the presence of a central coordinator. 
Finally, we apply our theoretical findings to study a novel multi¬ 
dimensional, convex-constrained model of opinion dynamics and 
a hierarchical demand-response scheme for energy management 
in smart buildings, extending literature results. 

I. Introduction 

In recent years, there has been an increasing interest in 
modeling and control of populations of agents that interact 
through a network. If the agents are noncooperative and profit- 
maximizing, these systems can be studied combining ideas of 
game theory and network analysis. Traditionally, the literature 
on network game theory has focused on games in which each 
agent behavior is influenced by its one-to-one interaction with 
the neighbors [1], [2], [3], [4], [5], When the size of the 
population becomes very large, however, the analysis of these 
models may become computationally intractable. Moreover, 
in many applications involving large populations of agents 
the well-being of an agent cannot be described as the overall 
result of individual two-player interactions, but should rather 
be modeled as a unique function depending on the agents 
strategy and on a quantity (either common to all agents or 
agent-specific) that depends on the aggregate behavior of the 
entire population. 

Examples of applications where the influence of the pop¬ 
ulation behavior is the same for all agents are demand side 
management in smart grids [6], [7], [8], charging coordination 
of plug-in electric vehicles [9], [10], congestion control [11] 
and economic markets [12]. This type of interactions can be 
modeled using the theory of (generalized) aggregative games 
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since the payoff of each agent is a function of its own strategy 
and of a common aggregator function, whose value depends 
on the strategies of all the players [13]. In this framework, 
to compute its best response each agent needs to know the 
aggregate of all the other players’ strategies; this may not be 
desirable when the population size is very large. One way 
to overcome this issue is to address large-size aggregative 
games by assuming a continuum of players that are influenced 
only by the population statistical distribution [14], [15]. These 
models can be efficiently analyzed by exploiting the so-called 
Mean Field (MF) approximation. Specifically, several schemes 
have been proposed in the literature to coordinate the agents 
behavior in a decentralized fashion, by using a statistical 
description of the population [14], [16], or by means of a 
central coordinator [9], [17]. 

Classical aggregative and mean field games are based on the 
assumption that all the agents are affected by the population 
via the same aggregator function. On the other hand, we 
consider here situations where the interaction, even though 
not one-to-one as in typical network games, still possesses 
a well determined structure. Examples where such structures 
appear are (generalized) quasi-aggregative games [18] where 
each agent has a different interaction function, leader-follower 
games [19] where some agents have more control authority 
or knowledge than others, games with local/agent-dependent 
cost functions [20], [21], games where the agents are allowed 
to interact only locally [8], [22], or games where the agents 
have different stubbornness [23], [24]. Here, we consider a 
setting in which each agent is influenced by the aggregate 
strategies of an agent-dependent subset of the population, 
which is defined by an underlying network. We refer to this 
type of games as Network Aggregative (NA) games. Among 
these, we restrict our attention to games in which each agent 
minimizes a quadratic cost function that depends on its own 
strategy and on a convex combination of the strategies of its 
neighbors, that we refer to as the neighbors’ aggregate state. 
Moreover, contrary to traditional (quasi-) aggregative games, 
we allow the aggregator function to be vector valued instead 
of scalar valued. 

For this class of systems, we are interested in sets of 
strategies where no agent has interest in unilaterally deviating 
from its own behavior, that is, we aim at characterizing the 
Nash equilibria of the NA game. Moreover, we analyze the 
dynamics arising in a population where each agent updates 
synchronously its strategy in response to the strategies of its 
neighbors. The simplest type of dynamics that we investigate 
is the so-called Best Response (BR) dynamics, where the 
agents adopt at every iteration the strategy that minimizes 
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their cost, given the current neighbors’ aggregate state. Our 
first technical contribution is the derivation of conditions on 
the cost function and on the network structure that guarantee 
convergence of the BR dynamics to a Nash equilibrium. For 
cases where the BR dynamics are not guaranteed to converge, 
we propose different strategy update schemes with memory 
that can be implemented by the agents in a distributed fashion 
and, under different conditions, ensure convergence to a Nash 
equilibrium. 

Secondly, we show how the setting of NA games can 
be used to steer a population of agents to an almost Nash 
equilibrium of a deterministic MF game by allowing multiple 
rounds of communications between two strategy updates. To 
this end, we use results from consensus theory to ensure that 
the agents can reconstruct the average of the strategies of the 
entire population by communicating with their neighbors. The 
advantage of this approach is that it does not require communi¬ 
cations with all the agents in the population (as in aggregative 
games) making our method scalable as the population size 
increases, nor the presence of a central coordinator or a statis¬ 
tical description of the population (as in MF games). The task 
of recovering a MF Nash equilibrium by allowing only local 
communications over a network has been already considered 
in the literature for the case of linear cost [8] or scalar state 
variables [25], via variational inequality approaches. Instead, 
we consider here a setting where each agent solves a multi¬ 
dimensional convex-constrained optimization problem with 
quadratic cost function. A key aspect that differentiates our 
approach from more standard distributed optimization schemes 
is that in our setting, at every iteration of the algorithm, each 
agent selects its strategy as the BR to its current estimate of 
the population behavior. This is a fundamental feature for the 
applicability of our schemes in a population of non-cooperative 
agents. We also notice that the inverse problem of recovering 
consensus via MF games has been studied in [26], [27]. 

The contributions of the paper are organized as follows. 

• As motivating application of NA games, we formulate in 
Section II a novel extension of the classical Friedkin and 
Johnsen model of opinion dynamics by allowing each 
agent to update its opinions regarding a collection of 
different topics, linked together by convex and compact 
constraints. Moreover, we suggest a distributed imple¬ 
mentation of a classic model of demand-response scheme 
for smart energy management, as motivating application 
for distributed MF control. 

• In Section III we define NA games and illustrate their 
relations with aggregative and MF games. In Section III-C 
we extend the definition of NA games by allowing 
multiple rounds of communications. 

• In Section IV we introduce the concept of aggregation 
mapping, generalizing the ideas presented in [17], and 
characterize the game-theoretical properties of the set of 
strategies computed as BR to its fixed points, for both 
NA and MF games. 

• In Section V we derive conditions guaranteeing the 
convergence of the BR dynamics and of newly introduced 
update schemes to a fixed point of the aggregation 
mapping, generalizing the results in [28]. 


• In Section VI we apply our theoretical findings to the 
applications introduced in Section II, extending previous 
literature results. 

Section VII concludes the paper and highlights several pos¬ 
sible extensions and applications. Appendix A presents some 
background definitions and results from operator theory; Ap¬ 
pendix B contains all the proofs of the technical results. 

Notation 

R, R>o, R>o respectively denote the set of real, positive 
real, non-negative real numbers; N denotes the set of positive 
natural numbers; 2N denotes the set of even numbers; Z 
denotes the set of integer numbers; for a, b £ Z, a < b, 
Z[a, b] := [a, b] (T Z. I. n denotes the n-dimensional identity 
matrix; 1„ denotes the n-dimensional vector with all entries 
equal to 1. For a given Q £ R nx ™, the positive definite 
notation Q >~ 0 (as well as ^=,^,=^) implies Q = Q T . 
For a given Q G R" xn , Q >- 0, we denote by TLq the 
Hilbert space R” with inner product (-,-)q : R” x R" -> R 
defined as (x,u)q := x T Qy , and norm ||-||q : R" —x R>o 
defined as ||a ;||q := sjx 1 Qx\ ||a:|| := ||at||/„. The projection 
operator in Hq, Proj^? : R" —x C C R", is defined 
as Proj^(a;) := argmin ygC ||a: — t/||g. A ® B denotes the 
Kronecker product between matrices A and B. Given a matrix 
P, we define P (J := [P]^. its element in position (i,j) 
and for any v £ N, Pjj denotes the element in position 
(i,j) of the matrix P". The matrix P is row-stochastic if 
£;£ Pij = 1 f°r all i £ Z [1, N\ and doubly-stochastic if P 
and P are row-stochastic. The matrix P is primitive if there 
exists h £ N such that P h is element-wise strictly positive. 
||P||q := sup I x € R" \ {0} j denotes the induced 

matrix norm. We denote by A(P) the set of eigenvalues of 
P, by p(P) := max] |A| | A € A(P)} its spectral radius 
and by cr max (P) its maximum singular value. If P 0, 
we denote by Ps 'y= 0 its principal square root. Given 
X C R”, A £ R rax " and b £ R ra , AX + b denotes 
the set {Ax + b £ R" | x £ X}; given X lt ..., X^ C R n , 
Xi xN := XiX. . .xX N , X%) :={^£tt € R" I 

Xi £ Xi , Mi £ Z[1,AT]}, conv{A’ i }^ 1 := \Xi £ 

R” | X r £ X t , Xi £ [0,1] Mi £ z[l,(V],£tt£ = 1}. 
Given two sets X t . X- 2 and a vector x \ £ X\ we define the 
premetric p(X 1 ,X 2 ) := sup^^ inf X2e * 2 ||ati - x 2 || and 
X 2 ) := inf^g^ ||a;i — x||. Given N vectors Vi £ R n , 
i £ Z [1, AT], we denote [iq;... ;uat] := e R^”. 

Given /,j:XC R” —x R, f(x) = 0(g( x)) denotes that there 
exists K > 0 such that f{x) < Kg{x) for all x£l 

II. Aggregative games over a network: Two 

MOTIVATING EXAMPLES 

A. Multi-dimentional constrained opinion dynamics in social 
networks with stubborn agents 

We first consider the problem of modeling how ideas, 
innovations or behaviors spread in a social network of N £ N 
agents [29], Generalizing the setting described in [30], we 
assume that each agent i £ Z [1, N\ has a vector x z £ [0,1]” of 
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opinions regarding n £ N topics. Each component x\ £ [0,1] 
represents the opinion of agent i about topic s £ Z [1 ,n\, 
where 0 represents an extremely negative and 1 an extremely 
positive opinion. We denote by x£ [0,1]" the initial 
opinion of agent i. Moreover, we specify the social network 
by the weighted adjacency matrix P £ M. NxN , where the 
element P l3 £ [0,1] denotes the relevance of the opinion of 
agent j to the decision of agent i. In the following we assume 
without loss of generality that ■ P t] - 1, for all i £ Z[1 ,N]. 
To describe the opinion dynamics, we consider a synchronous 
repetitive game where at every iteration k each agent i commu¬ 
nicates once with its neighbors Af l := {j £ Z[l, N] \ Pij > 0} 
and updates its opinion according to the optimization problem 

N 

argmin YV-Py Hz* - aV|| 2 ) + 6i\\x l - rr'( 0) || 2 

s.t. x i £ X\ (1) 

The cost function in (1) comprises two terms: the first one 
models the influence of the neighbors to the new opinion of 
agent i, the second one models the “stubbornness” of agent i 
about its initial opinion. Additional constraints on the agents’ 
opinions across the n topics, as for example the fact that the 
opinions regarding two topics should not differ more than a 
given threshold, and on their stubbornness can be encoded via 
the constraint set X 1 C [0,1]". The agents are assumed to be 
heterogeneous in the sense that the stubbornness parameter 
9i > 0, the constraint set X 1 and the weights may 

be different for every agent. Following the nomenclature used 
in [30], we refer to the agents for which X 1 = {a;^}, that 
is, agents that are not influenced by the neighbors, as fully 
stubborn , to the agents for which 0i = 0 and X 1 = [0, l] n as 
followers and to all the remaining agents as partially stubborn. 

In the absence of constraints, the solution to (1) for each 
topic decouples, hence we can consider n = 1 without loss of 
generality. The solution in this case is given by 

X (x^ ) = j+q- P ij xJ #(0) > 

which leads to the BR dynamics x]u +l ) = x l *(x = 

yzfg- p i 3 x \k) + i+e^ x \o) which is a particular case of 

the standard Friedkin and Johnsen model [31], with parameters 
A := diag ^ 1 _ | 1 6 , i ,..., x +g N ^ and W := P. For the case in 
which there is at least one partially stubborn agent it is possible 
to show, with the same argument used in [30], that the BR 
dynamics converge to a Nash equilibrium of the game in (1), 
that is, to a configuration where no agent has incentive in 
modifying its opinion given the opinions of its neighbors. If 
on the other hand, all the agents are followers, one recovers the 
standard DeGroot model [32], whose convergence properties 
have been exhaustively investigated using classical consensus 
theory. We note that, in this case, the Nash equilibria coincide 
with the right eigenvectors of the matrix P corresponding to 
the eigenvalue A = 1. 

As a corollary of the theory developed in this paper, 
we derive in Section VI-A conditions under which the BR 
dynamics converge to a Nash equilibrium in the presence of 



stubborn agents and generic convex constraints. Moreover, 
for the case in which the BR dynamics do not converge, we 
propose the use of alternative opinion-update rules that ensure 
convergence. To this end, we note that the cost function in (1) 
can be rewritten, up to constant terms that do not depend on 
x l , as 

J\x\a l ) := {l+e i )\\x i \\ 2 ^2{a i + e i f i {Q) ) T x\ (2) 

where a 1 = p ij xJ is the average opinion of the 

neighbors of agent i. Consequently, the game in (1) can be 
thought of as a game where each agent tries to minimize a cost 
function that depends on its own strategy x 1 and on a convex 
combination of the strategies of its neighbors a \ resulting in 
a NA game, as defined in Section III-A. 


B. Demand-response methods and mean field control via local 
communications 

As second motivating example, we consider a population 
of N £ N loads whose electricity consumption u l = 
[u \,..., ufj £ R t over the horizon T = Z [1, T] is scheduled 
according to the following demand-response scheme 

u l * (ct) := argmin ^ (p, \\u t - u\\\ 2 + p(d t )ut) (3) 
«e RT t £T V 

s.t. s t+ i = a l s t + 7 l u t Vi £ T 
[s(u);u] £ (S‘ xU^nCf 

where St = St(u ) is the state of the load at time t (e.g., 
its temperature in case of heating ventilation air conditioning 
systems [33], as detailed in Section VI-B, and thermostatically 
controlled loads [34] or its state of charge in case of plug¬ 
in electric vehicles [9], [10], [17]), s\ £ I is the given 
initial state, a 1 , 7 * £ K.\{ 0 } are parameters modeling the 
dynamics and the efficiency of load i, d t = 'f \ u\ £ ffi. 
is the population aggregate energy demand at time t, a := 
[cti;...; <7t] £ K T . The energy consumption u and state 
vector s{u) are constrained by the personalized sets U l C M T 
and S l C M T , respectively, and by the coupling constraint 
set C l C R 2T . The first term in the cost function of (3) 
models the curtailment cost that each agent encounters for 
deviating from its nominal energy schedule u\, according to 
the Taguchi loss function [35], Pi > 0 being a constant 
weighting parameter. The second term models the demand- 
response mechanism: the price that each agent has to pay 
for the required energy varies according to a price function 
p(dt) that is increasing in the total energy demand at time 
t. Since the price function depends on the average strategy 
d of all the players , that is a 1 = d for all agents, this is a 
deterministic quadratic mean field game, as described in [ 10 ], 
[16], [17]. The associated mean field control problem aims 
at designing a price vector p(d) such that the set of optimal 
responses {u l '*(d)}^L 1 possesses some desirable properties in 
terms of the MF game in (3). Approaches to solve this problem 
iteratively via a central coordinator have been proposed in 
[9], [10], [17], [33]. Here we propose a distributed algorithm 
with guaranteed convergence to an equilibrium price signal, 
by using local communications only. We also investigate the 
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relation of the resulting solution with the one obtained via a 
central coordinator. 

III. Game setting and previous work 
A. Constrained network aggregative games 

Consider a population of JV £ N heterogeneous agents, 
where each agent i £ Z[l,i\T] controls a decision variable 
x \ taking values in the set X 1 C R", and interacts with 
the other agents via a directed network (Figure 1). In the 



following, we specify the network by the weighted adjacency 
matrix P £ M. NxN , whose element P t) £ [0,1] denotes 
the strength (or relevance) of the communication from agent 
j to agent i, Py = 0 implying no communication. The 
aim of agent i is to minimize its individual deterministic 
cost J l (x l ,o l ) that depends on its own strategy x l and its 
aggregation a 1 := EjL i P ij x 3 with the strategies of its 
neighbors Af‘ := {j ^ i \ Pij > 0}. Specifically, each agent 
i £ Z[l, N] aims at computing the Best Response (BR) to the 
neighbors’ aggregate state a 1 

x^cr 1 ) := argmin J l (x*,cr*) (4) 

x i GX i 

= argmin J l (x\ P u x l + P ij x3 ) ■ 

Since the neighbors’ aggregate state a 1 is different for each 
agent and is specified by the network, we refer to this problem 
as a Network Aggregative (NA) game. In classical game 
theory, a set of strategies in which every agent is playing a 
best response to the other players’ strategies is called Nash 
equilibrium. In the aggregative case, the concept is similar: 
the population is at a NA Nash equilibrium if each agent 
has no individual benefit in changing its strategy, given the 
aggregation among the strategies of the neighbors. 

Definition 1 (Network aggregative Nash equilibrium): 
Given N cost functions { J 1 : R n x R" —» R}^ and a 
weighted adjacency matrix P £ M. NxN , a set of strategies 
{a:* £ X % C M n } i _ 1 is a NA Nash equilibrium for (4) if, for 

all i £ Z[l, N], 

J 1 p ii xj ) = min J 1 (y, P it y + p ij xj ) • D 

One of the main challenges in analyzing aggregative games, 
and games in general, is to characterize the evolution of the 
players’ strategies when the game is repeated iteratively. For 
this problem to be well defined one has to specify the update 
rule used by each agent i, at iteration k, to select its updated 


optimal strategy ) in response to the strategies of its 

neighbors. The simplest dynamics are obtained when, at every 
iteration, all the players synchronously compute their BRs to 
the current strategies of the neighbors, that is 

*(fc+i) : = **(*(*))» ( 5 ) 

then communicate and update the neighbors’ aggregate state. 
This scheme is known as “best response” dynamics or “my¬ 
opic” dynamics [18], since the agents do not take into account 
the past or future evolution of the game to update their 
strategies. One of the main aims of the paper is to derive 
conditions under which the BR dynamics converge to a Nash 
equilibrium of the original game as described in Definition 1. 

In this paper, we restrict our attention to NA games satis¬ 
fying the following assumption. 

Assumption 1 (Game setting): Each agent i £ Z[1,1V] has 
a convex and compact constraint set X 1 C R” and quadratic 
cost 

J‘(x l ,(7 l ) := qiX lT Qx l + 2 (Ca l + Cj) T x l , (6) 

where x l ,a l £ R n , Q >- 0, qi > 0, C £ R" x ” and c, £ R n . 

The weighted adjacency matrix P € Dl NxN is row stochastic, 
that is, EyLi P ij = 1, Vi £ Z[1 ,N], □ 

Note that the cost function in (6) combines two different terms: 
a quadratic cost specific to the individual and an affine term 
associated with the neighbors’ aggregate state a 1 . Under this 
assumption, the minimizer in (4) is unique. 

B. Aggregative games, deterministic mean field games and 
mean field control 

If the network is fully connected, that is, if a 1 = d = 
jj Ef = i x3 G R” for all i, then the corresponding NA game 
is a particular case of aggregative game with aggregator d. 
Aggregative games, in turn, have a strong connection with 
deterministic Mean Field (MF) games. The latter term is used 
in the literature to denote a game, played among a large 
population of N heterogeneous agents, where each agent is 
influenced by the statistical distribution of the strategies across 
the population. If each agent is influenced by the first moment 
of the distribution only, deterministic MF games coincide with 
aggregative games with aggregator d and hence with NA 
games over fully connected networks. Mathematically, in all 
these games each agent i £ Z [1, N] aims at computing its BR 

Xfrid) := argmin J l (x l ,d) (7) 

to the aggregate strategy d of all the players. 

Definition 2 (MF Nash equilibrium [17, Definition 1 ]): 
Given N cost functions { J 1 : R" x R" -) R}iE an d £ > 0, 
a set of strategies {x* £ X 1 C R”}^ 1 is a MF e-Nash 
equilibrium for (7) if, for all i £ Z[1,1V], 

Ji (^> ~N EjLi x °) < mm J 1 (y, p (y + x^ ) + e. 

(8) 

It is a MF Nash equilibrium if (8) holds with e = 0. □ 
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One of the objectives of mean field control theory is to study 
conditions under which it is possible to steer the population to 
such Nash equilibria, by using global macroscopic incentives 
only, see for example [8], [9], [17]. Let 

x l *(z) := argmin J l (x, z) (9) 

xGX i 

be the optimal solution that agent i would compute, according 
to the cost function J l given in (6), in response to a fixed 
global incentive 2 and consider the aggregation mapping A : 

R” (if 1 c R", defined as 

***(-*)■ ( 10 ) 

For the quadratic cost function considered in (6), it is proven 
in [17, Theorem 1] that A in (10) has a fixed point z = A(z) 
and the set of strategies {a £ A?*} is an ejv-Nash 
equilibrium for (7), with £jv decreasing to zero with rate 
as the population size N grows. To find such a fixed 

point, and hence steer the population to an e-Nash equilib¬ 

rium, one can use an iterative scheme among the agents and 
a central coordinator where, at every iteration step k , the 
central coordinator broadcasts a reference signal Z(k) to the 
population and each agent i responds by computing its optimal 
strategy := x z *(z(k))- The central coordinator then 

collects the average A(z^) of such strategies and updates 
the reference signal according to z^k +n = A(z^))) 

[17, Algorithm 1]. The feedback mapping <b :■(■■') can be 
selected to guarantee convergence, depending on the regularity 
properties of the aggregation mapping A. Suitable iterations 
and sufficient conditions on the matrices defining the cost 
in (6) are given in [17, Corollary 1], For example, if A is 
a contraction mapping (see Definition 4 in Appendix A) then 
the Picard-Banach mapping <1> P ~ B ( 2 ( fc ), A(z^))) -A-{ z (k)) 

ensures convergence to the unique fixed point of the aggrega¬ 
tion mapping A. 

Note that the centrally coordinated solution detailed above 
cannot be used to steer the agents to an equilibrium in NA 
games since, in this case, the signal a 1 is different for every 
agent. On the contrary, we show below that the distributed 
algorithms derived for NA games can be used to approximate 
the solution of MF games. For this purpose, we define one 
last class of games: NA games with multiple rounds of 
communications. 

C. Multiple-communication network aggregative games solve 
the mean field control problem 

The solution to the MF control problem outlined in Sec¬ 
tion III-B relies on the presence of a central coordinator that 
at every iteration computes the average signal A(z (k)) and 
broadcasts the reference signal Z(k+i) to the population of 
agents. To solve the original MF problem by means of local 
communications only, we consider a scenario where each agent 
i £ Z [1,7V] tries to recover the overall population average d 
by communicating vgN times with its neighbors; we indicate 
the resulting local estimate by a l u £ R". Formally, each agent 
i £ Z [1 , N] approximates the optimization problem in (7) 


with 

x bv(K) '■= argmin J z (x, <r®), (11) 

xGX i 

where crj, := an d Pfj denotes the element (i.j) 

of P v . Definition 1 on NA Nash equilibrium can be extended 
to the case of multiple communications as follows. 

Definition 3 (Multiple-communication NA Nash equilibrium): 
Given N cost functions { J l : R" xR"-> R}^, a weighted 
adjacency matrix P £ R NxN and a fixed number of 
communications u € N, a set of strategies {x* £ X i } ! * =1 is a 
NA Nash equilibrium for (11) if, for all i £ Z[l, N], it holds 

J l Ef=i = mm J‘ (y, P? x y + £ P%&) . □ 

We show in the following how the update rules derived to 
guarantee convergence to a NA Nash equilibrium of the game 
in (4) can be applied also to the multiple-communication NA 
game in (11) to steer the agents’ strategies to a MF e-Nash 
equilibrium of the MF game in (7), in a distributed fashion, 
for large N and v. 

IV. On network aggregative and mean field 
Nash equilibria 

To address the tasks described in Sections III-A and III-C, 
we solve the following problems. 

• Problem 1: Given a single (u = 1) or multiple (v > 1) 
communication NA game, derive update rules and con¬ 
ditions under which the agents’ strategies converge to a 
NA Nash equilibrium, as described in Definition 1 and 3, 
respectively; 

• Problem 2: Given a deterministic MF game as in (7) over 
a population of size N and a network with weighted 
adjacency matrix Pm, derive update rules and condi¬ 
tions under which the agents’ strategies converge, in 
a distributed fashion, to a MF e-Nash equilibrium, as 
described in Definition 2, with e decreasing as N and 
v increase. 

To solve these two problems, we start by defining the optimal 
response of an agent to a fixed signal z l £ R" as 

x i *(z i ) : = argmin J l (x i ,z i ). (12) 

x i ex i 

This mapping differs from the BR mapping since the second 
argument in the cost function is not the neighbors’ 

aggregate state a 1 (that in general could depend on x l ), but 
is a fixed signal z l (that does not depend on the optimization 
variable x l ). Moreover, the mapping in (12) differs from the 
one in (9) since the signal z l is possibly different for each 
agent. Let z := [z 1 ;...; z N ~\ £ M. Nn be a vector of (possibly 
different) signals for each agent and define the mapping x * : 
R Nn X lxN as 

x*(z) := [x 1 *^ 1 );.,. ;x N *(z N )] £ R Nn , (13) 

whose sub-vectors are the optimal strategies computed by each 
agent i according to the local signal z'. The mapping x* 
in (13) can be used to define an extended aggregation mapping 
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Ay that, given a vector z, returns the updated estimates of the 
population average, after one optimization and v communica¬ 
tion steps. Formally, A v : R Nn (P v ® I n )X lxN C ffi . Nn 
is defined as 


A u (z) 


' Ai(z) - 


' £7=1 i^'V) ’ 

. A?(z) _ 


_ £f =1 PV j V) . 


(. P v ®I n )x*(z) =:T v x*(z). 


(14) 


A. Problem 1: Nash equilibria of single and multiple- 
communication network aggregative games 

In the following theorem we show that the fixed points 
of the aggregation mapping A,, can be used to find a Nash 
equilibrium of the NA game with v communications, for any 
population size N, under the following assumption on the 
network structure. 

Assumption 2 (Graph property): For the considered num¬ 
ber v £ N of communication steps and the given population 
size N. the weighted adjacency matrix P satisfies Pf = 0 for 
all i £ Z[l, N}. □ 

Remark 1: Assumption 2 is equivalent to the absence of 
cycles of length v in the graph associated with P [36, 
Theorem 4.1]. In the case of single-communication NA games, 
that is if v = 1 as defined in Section III-A, Assumption 2 
is equivalent to the absence of self-loops, which means that 
each agent plays against the average of its neighbors, itself 
excluded. If Pu/qt is the same for each agent i £ Z[l,iV] 
this assumption can be introduced without loss of generality 
by redefining the Q matrix of the cost function ./' in (6). Note 
that the absence of self-loops is necessary but not sufficient 
when v > 1. 

Assumption 2 guarantees that the neighbors’ aggregate state 
a/ computed by each agent i does not depend on its own 
strategy P. As a consequence, the BR mapping (11) coincides 
with the optimal response (12) to the signal z l := a/ = 
Pij xj = Therefore, if each agent is 

playing the BR to the i-th sub-vector z l of a fixed point 
z = [z 1 ;... , z N ] of the aggregation mapping «4.„, then the 
new vector of neighbors’ aggregate states coincides with z. 
This observation can be formalized in the following theorem. 


Theorem 1: Under Assumption 1 the mapping A„ in (14) 
admits at least one fixed point z = A.,,(z). If also As¬ 
sumption 2 holds and z is a fixed point of A„, then the 
set of strategies {a:** (z*)}i_ r with x l * as in (12) for all 
i £ Z[l, N ], is a NA Nash equilibrium for (11). □ 


Note that Theorem 1 implicitly ensures the existence of at 
least one NA Nash equilibrium for all NA games satisfying 
Assumptions 1 and 2. Moreover, under these assumptions, if 


the vector 


' v ( k ) ’ 


T N 

T v (k) 


of neighbors’ aggregate states 


defined by the BR dynamics in (5) tends, as k —> oo, to a fixed 
point of the mapping A,,, then the agents’ strategies converge 
to a NA Nash equilibrium. The derivation of conditions under 


which this happens and of alternative update rules is postponed 
to Section V. Finally, we note that MF games are NA games 
over a complete network with the same self-weight P lt = 1/N 
for all agents. If q, = q for all i, these games can be 
reformulated in a form compatible with Assumption 2, as 
suggested in Remark 1. Theorem 1 then guarantees that a MF 
Nash equilibrium exists (as shown also in [17, Proposition 1]) 
and can be obtained in a distributed fashion. The drawback 
is that, to this end, communications over a complete network 
are required. Since in most cases this is not desirable in the 
following section we derive conditions under which almost MF 
Nash equilibria can be recovered using local communications 
only. 

B. Problem 2: Distributed solution to the mean field control 
problem 

To address Problem 2, we start by analyzing the relation be¬ 
tween the extended aggregation mapping A,, in (14), obtained 
via v communications over the network P, and the aggregation 
mapping A : 1^" -)> (±1 N 1% <g> I n )X lxN , 

A(z) := <g> I n ) x*(z) =:Xx*(z), (15) 

that arises out of a complete network. We assume the following 
uniform bounds on the constraint sets X' and on q,. 

Assumption 3 (Compactness): There exists a compact set 
X C R" and q, q > 0 such that, for all population sizes N, 

conv J C X and q < qt < q for all * £ Z [1, N], □ 

For the game in (11) to approximate the MF^game in (7), 
we need to ensure that lim^oo &(, = d = xf that is 

the agents asymptotically reach consensus on the population 
average. To this aim we introduce the following assumption. 

Assumption 4 (Asymptotic average consensus): For all 
population sizes N, the weighted adjacency matrix P = Py 
satisfies lim,,-^ P v = -Tljvljv- Equivalently, P is primitive 
and doubly stochastic. □ 

Remark 2: Let us prove the equivalence of the two state¬ 
ments in Assumption 4. If lim u ^ fOQ P 1 ' = then 

there exists h £ N such that max, t j P-'^ — 1/N\ < 1/N. 
Consequently, there exists an h > 0 such that P h is element¬ 
wise strictly positive, which means that the matrix P is prim¬ 
itive. For a primitive and row stochastic matrix, by Perron- 
Frobenius theorem [37, Lemma 4], lim,,-^ P 1 ' = 1nw t , 
where it: 1 is the left eigenvector of P relative to the dominant 
eigenvalue 1 and normalized so that w t 1at = 1. It follows 
that lim^P" = w T = ^1at T <S=> 1at T P = 

1 n t ^ P is doubly stochastic. 

Note that Assumptions 2 and 4 both regard the matrix P" 
but are used to address two different problems and they are 
conceptually different. Assumption 2 imposes /”' = 0 for the 
fixed value of v considered in the NA game under analysis. 
Assumption 4, on the other hand, is needed when one wants 
to use multiple-communication NA games to approximate 
the solution to a MF game. Consequently, what matters is 
the asymptotic behavior of Pf, which should tend to the 
contribution that each agent has in a MF game, i.e. 1/N. 
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Lemma 1: Under Assumption 1, the mappings A in (15) 
and Au in (14), for all v G N, are continuous and have at 
least one fixed point. If additionally Assumption 4 holds, then 

lim^-Hx) sup zeK jvn II Av(z) - A(z )II =0. □ 

We next show that, by choosing v large enough, any fixed 
point of A„ is arbitrarily close to a fixed point of A. 

Lemma 2 (Fixed point sets): Let the sets of fixed points of 
A in (15) and A v in (14) be T := {z G R Wn | z = A(z)} 
and T v := {z G R Wn | z = *4„(z)}, respectively. If As¬ 
sumptions 1 and 4 hold, then for all e > 0 there exists v > 0 
such that p(F v ,F) < e for all v > v. □ 

We are now ready to state our main theorem regarding the 
solution to Problem 2. 

Theorem 2: Suppose that Assumptions 1, 3 and 4 hold. For 
all £ > 0 there exists N such that, for all N > TV, there exists 
v > 0 such that, for all v > v, if z is a fixed point of A v 
in (14), then the set of strategies {a; 1 * with x' 1 * as 

in (12) Vi G Z[l, N], is a MF £-Nash equilibrium for (7). □ 

We emphasize that, contrary to the case of NA games, where 
the fixed point of A v leads, under Assumption 2, to a NA Nash 
equilibrium for finite v and finite N, in the case of MF games 
a MF Nash equilibrium is recovered only asymptotically. This 
is due to the fact that, first, the agents are required to almost 
reach consensus on the population average (hence v should be 
large enough) and second, the population average depends on 
the strategy of agent i with contribution proportional to 1 /N 
(hence N should be large enough). However, for any desired 
£ > 0, the proof of Theorem 2 allows one to derive lower 
bounds on N and v in order to guarantee that {a;** (z*)} is 
a MF £-Nash equilibrium. We note that the minimum number 
of required communications v can be computed in a distributed 
fashion (Appendix C) and depends on e but also on the 
population size N and on the network P. In case of symmetric 
networks this dependence can be further specified in terms of 
the spectral properties of P. 

Assumption 5 (Spectral properties): For all population 
sizes N the weighted adjacency matrix P = P Xr is symmetric. 
There exists p, G [0,1) such p N := max AeA ( Piv )\ v { 1 } |A| < p 
for all N. □ 

Corollary 1: Suppose that Assumptions 1, 3, 4 and 5 hold. 
If z is a fixed point of A„ in (14), then the set of strategies 
{a: 1 * (z*)}^ V _ 1 , with x l * as in (12) for all i G Z[l, N], is a 
MF £-Nash equilibrium for (7), with e = 0( ^ + \fNp v ). □ 

In other words. Assumption 5 allows us to derive an upper 
bound on £ that is composed by two terms: one that decreases 
linearly in 1/N, as for the MF control solution with central 
coordinator [17, Remark 3], and one that, for any fixed N, 
decreases exponentially fast on the number of communication 
steps v. According to this bound, the number of communica¬ 
tion steps v required to achieve a fixed tolerance £ increases 
at most logarithmically in the population size N. 

Assumption 5 is satisfied, e.g., by the degree-normalized 
adjacency matrices of any family of d- regular undirected e- 
expander graphs [38, Definition 2.2 and Example 2.2], In fact 


Fn < 1 — 7^2 =: p [38, Theorem 2.4], [39, Theorem 2], 

V. Iterative schemes and fixed point convergence 

We now return to update schemes that, under different 
conditions, will ensure convergence to a fixed point of A,,. 


Algorithm 1: Memoryless update rule (BR) 

Initialization. Set k 4— 1, choose an initial reference . 
for every agent i. 

Iterate until convergence. 

Optimization step: each agent i computes its optimal 
strategy with respect to the reference z l 

x l * G- argmin J l (x, zL); 

Communication step: each agent i updates its neighbors’ 
state by communicating v times 

<Tq g- x i *\ 

for s = 0 to s == v — 1 do 

<+i <“ Ejlr p ij<> 
end 

and updates the reference 

4+U fT -’ 

k k + 1 ; 


The simplest update rule (Algorithm 1) is the one where 
each agent selects as strategy its optimal response to the 
current neighbors’ aggregate state, that is = c+ ^ := 

Pu x \k) + Pij x {k)- Mathematically, we can describe an 
iteration of this algorithm as z^+i) = A v (z^))- Under 
Assumption 2, the optimal response coincides with the BR, 
hence Algorithm 1 coincides with the BR dynamics. 

To allow convergence under more general conditions, we 
consider extensions of Algorithm 1 where at every step k 
each agent i computes its optimal response to a filtered 
version of a' v (k y instead of myopically computing the BR. 
Specifically, we assume that the agents perform G [0, v\ 
communication steps before updating their strategies and 
V 2 = v — v\ communications afterwards. Let Z(^) be the 
vector of local signals at the beginning of step k. After the 
first ;/| rounds of communications agent i obtains the averaged 
value Al l 0 (z^) := Yj P ij z [k) ( where the two subscripts 
of Al o re f er to the number of communication steps before 
and after the optimization step). Based on this, it computes its 
optimal strategy x l A := x l *(Al 1 o( z (k))), an d then commu¬ 
nicates the remaining v 2 times with its neighbors, obtaining 
+4 g 2 (^«) := Finall y’ it updates 

the local signal z l using a feedback mapping that filters 
the current aggregate state A l Vl V2 (z^)) with the previous local 

signai z\ ky that is z\ k+1) = (4)>^ti, V2 («(*)))• since the 

agents use information about the previous signal, we refer to 
Algorithm 2 as update rule with memory. 

Let $ : x R*" -G be the extended feedback 

mapping, defined as 

3>{z,A„ uV2 ) := [<S>(z\Al u „ 2 y,...-A(z N ,A^ 2 )\ , (16) 

and note that, by construction, it can be computed in a 
distributed fashion. An iteration of Algorithm 2 can then be 
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Algorithm 2: Update rule with memory 
Initialization. Set k 4— 1, choose an initial reference 
for every agent i, a feedback mapping {and 
^ 1,^2 £ Z> 0 , v\ + vi = v. 

Iterate until convergence. 

Communication step 1: each agent i updates its estimate 
by communicating Vi times 

^o.o z \ky 

for s = 0 to s = v i — l do 
Ai . sp N p. . A3 . 

*^5+1,0 ^ = 1 r 'i'3^ x s, 0’ 

end 

Optimization step: each agent i computes its optimal 
strategy with respect to the current estimate A z v 0 

x l * 4 — argmin J l (x, A Vl 0 ); 

x£X i 

Communication step 2: each agent i updates its estimate 
by communicating v 2 times 

3^,0 

for s = 0 to s = v 2 — I do 


A 


Y.j-1 PiiALr 


v . -s- f 1 


end 

and updates the reference 

Z (k+1) ( Z (k) » ,^2 


k <— k + 1; 


described by = <fr(z(fc),«4 j/ 1)I/2 (z(fc))). As candidate 

feedback mappings, we consider different classes of fixed 
point iterations from operator theory. The simplest feedback 
mapping is the Picard-Banach iteration [40, Theorem 2.1] 


Theorem 3: Under Assumption 1, the following iterations 
and conditions guarantee that the sequence in (20) converges, 
for any initial vector z 0 £ R Nn , to a fixed point of A„ in (14). 




eq. 

(^ 1 ,^ 2 ) 

cost (Vi) 

network 

1. 

$P-B 

(17) 

(0,u) 

Mi >- 0 

ll^ll < 1 

2. 

q> K 

(18) 

(0,u) 

Mi V 0 

\\P\\<1 

3. 

$p-b 

(17) 

(v/2, v/2) 

-qiQ =4 C -4 0 

P =P T 

4. 


(19) 

{y/2, v/2) 

Cy 0 

P = P T 


The mapping A„ has a unique fixed point in case 1. □ 


The assumption P = P T corresponds to having an undirected 
graph with symmetric weights. The assumption ||P|| < 1 
is more involved. In general, the 2-norm coincides with the 
largest singular value of the matrix P. In case of an undi¬ 
rected symmetric graph, the singular values coincide with the 
magnitude of the eigenvalues, hence the 2-norm results in the 
spectral radius of the matrix P, which under the assumption 
of row-stochasticity is 1. Hence the condition ||P|| < 1 is 
always satisfied for symmetric graphs. In the case of directed 
graphs one can use Holder’s inequality ||P| I < a/||P||i||p||oo 
to prove that any doubly-stochastic matrix P satisfies the 
assumption. 

It follows from the proof of Theorem 3 that whenever 
<I> P B is applicable so are $ K and <1> M , and whenever <l> K is 
applicable so is $ M . Which one among the possible choices of 
feedback mappings provides the best convergence performance 
is in general problem dependent; known convergence rates are 
available for $ P ~ B [40, Theorem 2.1]. 

VI. Network applications 


z (k+i) = $ p B {z (k) ,A Vl , V2 (z (fc) )):= A VuV2 (z (fc) ) . (17) 

Using this mapping and setting (izl,^) = (0, 1 /) it is easy to 
see that Algorithm 1 is a particular case of Algorithm 2; from 
now on we therefore restrict our attention to the latter algo¬ 
rithm. More general fixed point iterations are the Krasnoselskij 
iteration [40, Theorem 3.2] 

^ (^(fc)> A v , 1i i, 2 (z(fc))) . (1 A)z(£q T XA L , l ^2 (-^(fc)) 

(18) 

with A £ (0,1), and the step-dependent Mann iteration [40, 
Definition 4.1] 


(^(fc) ’ (^(fc))) • (1 “I - ^kAiy llL > 2 

(19) 

where the sequence (ctfc)^ 1 is such that a k G (0,1) Vfc > 0, 
linifc-j-oo oi k = 0 and J2kLi a k = 00 ( e -g-> a k = 1/Ar). 

The following result provides conditions on the cost func¬ 
tions and on the network structure under which the sequence 
of vectors used to compute the optimal responses, 

(MLo( z (*o); A vuo( z (k))])™ =1 , (2°) 


converges as k tends to infinity to a fixed point of the 
aggregation mapping A u in (14). As a consequence, 
converges to the desired Nash equilibrium configuration, ac¬ 
cording to Theorem 1 and Theorem 2. Let 


Mi 


qiQ -C 
~C T qiQ ’ 


i £ Z[1,7V] . 


( 21 ) 


A. Multi-dimentional constrained opinion dynamics in social 
networks with stubborn agents 

The constrained opinion dynamics problem introduced in 
Section II-A can be rewritten as a single-communication (u = 
1) NA game, as defined in (4), by using the cost function (6) 
with qi := (1 + 9i),Q = I n ,C = -~I n , c = -9 t x ( 0) and 
constraint set X 1 . Since < 7 * does not depend on x l , Pa = 0 
for every i and Assumption 2 is satisfied. Consequently, by 
Theorem 1, if z is a fixed point of the mapping A\ in (14), 
then the set of strategies x l *(z l ) is a NA Nash equilibrium. 
The following result shows that if 9i > 0 for all i £ Z[1,7V] 
and ||P|| < 1, the BR dynamics converge to the unique fixed 
point of A\. If some followers are present in the population, 
then a Nash equilibrium configuration can be reached using 
the update rule with local memory. 

Corollary 2: Suppose that Assumption 1 holds. The fol¬ 
lowing iterations and conditions guarantee convergence of the 
opinions computed according to Algorithm 2, from any initial 
configuration, to a NA Nash equilibrium for (1). 

eq. {vi,V 2 ) cost (Vi) network 

1 . $ p - B (17) (0,1) 9i> 0 ||P||<1; 

2. $ K (18) (0,1) ||P|| < 1. □ 

Note that Algorithm 2 associated with the feedback mapping 
<f> p ~ B and (v \, 1 / 2 ) = (0,1) corresponds to the BR dynamics 
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Fig. 2: Three different network topologies: fully connected, 
directed ring and undirected small world. The top line is for 
a population of N = 10, the bottom for N = 100 agents. 
The small world networks have been generated by adding 
undirected shortcut links to the undirected ring topology, each 
with probability 0.3 [41]. The weights have been assigned 
so that the resulting P matrix is doubly stochastic. The 
corresponding matrices verify ||P|| = 1. 


(Algorithm 1) that are typically studied in the literature of co¬ 
ordination games. The use of the feedback mapping 4> K , on the 
other hand, extends the previous setting to the case of agents 
with memory. This additional feature allows one to recover 
a Nash equilibrium in the more general case of a population 
possibly containing both stubborn agents and followers. Note 
that the conditions of Corollary 2 are only sufficient, therefore 
there might be cases where the BR dynamics converge even 
in the presence of followers. It is however possible to find 
cases where the BR dynamics do not converge; an example is 
a population of followers communicating through a directed 
ring network. 

To investigate the performance of the two schemes we 
consider a case study where each agent i has n = 2 opinions 
x l = [jrl, # 2 ] , regarding two different topics, taking values 
in X 1 := {[xi,X2] V | ||xi — X 2 II 2 < 0.3} and is either 
a follower or partially stubborn with 0 t = 1. Figure 3 
reports the number of iterations required to reach convergence 
as a function of the population size N for two different 
compositions of the population and three different network 
topologies, illustrated in Figure 2. These simulations show that 
the convergence speed depends only mildly on the population 
size, making our approach scalable. 


B. Demand-response methods and mean field control with 
local communication 

By assuming an affine dependence of the cost on the 
population average, that is, 

p(a t ) := Xdt+Po, A > 0, (22) 

the game in (3) can be rewritten as the MF game in (7) with 
x l = u l and cost function as in (6) with cr* = d, qi = pi , 
Q = It,C = ^It , c = ^ — Ox 1 . Note that, since s(u) is 
an affine function of the input u, the constraint [s(u);u] £ 
( S 1 x U l ) IT C l can be rewritten as a unique constraint on the 
input, u l = x l £ X\ that is convex and compact if U'.S'.C 



Fig. 3: Average number of iterations (solid line) and 90% 
confidence intervals as a function of the population size N 
for the three different network topologies of Figure 2. The 
plot on the top refers to a population where only partially 
stubborn agents are present and update their opinion using 
the BR scheme (4 |P B ), while the plot on the bottom refers 
to a population composed by half partially stubborn and half 
follower agents, using the scheme with memory (<1> K ). In each 
case, 50 different networks and populations were simulated 
with initial opinions chosen according to a uniform distribution 
in [0, l] 2 . For each stubborn agent the initial opinion was 
projected on X' to guarantee feasibility of the constraints 
in (1). The stopping criterion is || zr^) — z (k-i) ||oo < 10 -5 . 


are convex and compact. The previous theory can be used to 
find a MF e-Nash equilibrium using local communications. 

Corollary 3: Suppose that Assumptions 1, 3 and 4 hold, 
and let p(d t ) be as in (22). The following iterations and 
conditions guarantee convergence of the strategies computed 
according to Algorithm 2, from any initial point, to a MF 
£jv,i/-Nash equilibrium for (3). 




eq. 

Oi,^) 

cost (Vi) 

network 

1 . 

$p-b 

(17) 

( 0 , 1 /) 

Pi ^ 2 

\\p\ 1 < i; 

2. 

$ K 

(18) 

(0 ,v) 

Pi > | 

\\p\\ < i; 

3. 


(19) 

{v/2,v/2) 


p = p t . 
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The model given in (3) can be used to describe demand- 
response methods for heating ventilation air conditioning 
(HVAC) systems in smart buildings, as suggested in [33], by 
selecting pi = O')f, where 0 > 0 is the cost coefficient of 
the Taguchi loss function and 7i > 0 specifies the thermal 
characteristic of the HVAC system. In [33, Theorems 1, 2] 
it is shown that, for N > 3, if 'f = 7 > 0 for all i, 
= [<in;<ax] C R" with < in , G R n , and A < 
then the Nash equilibrium is unique and can be computed 
using a control algorithm involving a central coordinator. 
Corollary 3 proves that Algorithm 2, on the other hand, can 
be used to find an e-Nash equilibrium in a distributed fashion, 
under less stringent conditions. Firstly, it allows arbitrary 
convex constraints u l G U\ instead of box constraints, hence 
including the case of joint stage constraints, ramping and 
more general dynamics. Secondly, convergence is guaranteed 
using only local communications over a network that satisfies 
||P|| < 1 and Assumption 4, instead of requiring the presence 
of a central coordinator. Thirdly, it demands the less stringent 
assumption A < 20 h for all i (or no assumption at all if 
additionally P = P T ), making the problem scalable for large 
population sizes. As a drawback, a MF e-Nash equilibrium 
is reached instead of an exact one, with e arbitrarily small 
for large populations. As a particular case we consider a 
hierarchical communication structure that models the fact that 
groups of buildings are managed by the same company. For 
simplicity, let us assume that there are M companies and 
each one manages B buildings, for a total of N = MB 
buildings. At every communication step the managers compute 
the aggregate power demand of their buildings, then commu¬ 
nicate among each other using a network Pm G ]R MxM and 
finally compute the price signal for their buildings. With the 
convention that buildings controlled by the same manager are 
grouped together in the extended vector and that the manager 
is the first agent of the corresponding block, the above scheme 
corresponds to the overall weighted adjacency matrix 

/ r 1 0 ... 0 

P = {lM 0 : i : : 

V Li 0 ... 0 

V_ v _ 

price dispatch 

= Pm 0 5 Islfj- 

Lemma 10 in the Appendix, proves that if Pm satisfies the 
connectivity conditions of Assumption 4 then also the matrix 
P in (23) does and that ||Pm ||2 < 1 implies ||P|| < 1 . 
Moreover, if Pm = Pm then P = P T . It follows from 
Corollary 3 that such hierarchical communication structure can 
be used, instead of a central coordinator, to steer the population 
to an e-Nash equilibrium. 

In Figure 4 we report the number of iterations required to 
reach convergence, as a function of the population size N, 
using Algorithm 2 with the mapping <I> M and different number 
of communications, and we compare it with the performance 
of the traditional centrally coordinated MF control scheme. 
As in the opinion dynamics case study, the required number 
of iterations is almost independent on the population size. 
Moreover, one can notice that already for small values of v 


(Pm 0 Ib) hi 1 


l/B 

0 


l/B 

0 


manager comm. 


local aggregation 


(23) 



Population size 


Fig. 4: Comparison between the MF control approach with 
central unit and the hierarchical distributed approach with 
M = 5 and v = 2,10,50. The matrix Pm corre¬ 

sponds to a symmetric ring network. The plot on the top 
shows the required average number of iterations needed to 
reach convergence (and the 90% confidence intervals), the 
one on the bottom the average cost improvement e, v := 
maxjgf! J l — J **, where J l := J ^ x *, ^ and 

■P* := min x i e *i J (x\ j) (x* + that an a 8 ent 

can achieve by unilateral deviations, both as a function of the 
population size N. We set A = 2 and Ojf = 0.1 for all i. We 
consider an horizon of T = 24 hrs and we assume a baseline 
energy consumption do as illustrated in [9, Figure 1], We set 
the baseline energy price to p 0 := Ado- The average is com¬ 
puted over 10 different populations with prescheduled energy 
consumption x l uniformly sampled in [0, l] 7 and constraint 
set X 1 := {a: G R| 0 | Ylt=i x t = Et=i x u x t = 0 if f ^ 
[ T L,v T cnd] }> with Etart uniformly sampled in {1 ,T}, T* nd 
uniformly sampled in {T^t+l, T}. To guarantee convergence, 
we use <1> ; M and v\ = v% = |. The stopping criterion is 
||w4. l /(^4. l /i J o(^'(fc))) *^- 1 / 1 , 0 h{k)) II 00 L: 10 1 . 


the solution to the distributed approach performs similarly to 
the solution obtained with the central coordinator. The higher 
the number of communications is between two optimization 
steps the smaller is e, as guaranteed by Theorem 2, and the 
fewer iterations of Algorithm 2 are required. 
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We conclude this section by noting that the model given 
in (3) can be also used, by setting pi sufficiently small, to 
compute the optimal charging strategy for large populations 
of Plug-in Electrical Vehicles (PEV) [9], [10], Under the 
same conditions of [17, Corollary 3], Corollary 3 (point 3.) 
allows to recover a MF e-Nash equilibrium using a symmetric 
network P instead of a central coordinator. 

VII. Conclusion and Outlook 

We have considered NA games for populations of agents 
with different individual behaviors, constraints and interests, 
and affected by the aggregate behavior of their neighbors. 
We have characterized the Nash equilibria of such problems 
and studied the convergence properties of the BR dynamics. 
For cases where it was not possible to guarantee convergence 
of the BR dynamics, we have proposed new strategy-update 
schemes. Finally, we have shown how the NA setting can be 
extended, by allowing multiple rounds of communications, to 
provide a distributed solution to the MF control problem. 

Our technical results are derived for agents that update their 
strategies synchronously and over a fixed network. As future 
work, we believe it would be interesting to study whether 
similar convergence results can be achieved via asynchronous 
updates and time-varying or random communications [42], as 
for example using gossip-based communication schemes. For 
the multiple communication case, known results from consen¬ 
sus theory could be applied to guarantee that the convergence 
requirement cr* (k j —> a is achieved asynchronously. 

The concept of social global optimality [43] has not been 
considered in this paper. It would be valuable to characterize 
the influence of the network structure on the global properties 
of the associated NA Nash equilibrium. In other words, 
one may favor networks with given properties in other to 
coordinate the agents to a desired emergent behavior. 

As in traditional aggregative games we have addressed a de¬ 
terministic setting; a valuable extension would be a stochastic 
setting where, for instance, the parameters of each agent are 
extracted from a probability distribution and/or a random input 
enters in the dynamics, as assumed in classical MF games [14, 
Section V], [14, Equation 2.1], 

Applications of our methods and results include distributed 
control and game-theoretic coordination in network systems. 
Among others, application domains that can be further ex¬ 
plored are multi-dimensional opinion dynamics on social 
networks [44], [45], [46], [47], distributed dynamic demand- 
side management of aggregated loads in power grids [8], [33], 
congestion control over networks [11], synchronization and 
frequency regulation among populations of coupled oscillators 
[48], [49], 

Appendix A 

The main idea behind the proof of Theorem 3 is to derive 
sufficient conditions on the matrices defining the cost func¬ 
tion (6) and on the network P to guarantee that the extended 
aggregation mapping A.,, (■) possesses one of the regularity 
properties listed in the following definition. 


Definition 4 (Regularity properties): Consider the Hilbert 
space Hs defined by the matrix S € R raxn , S >- 0. A mapping 

/ : R" —► R" is 

1) a Contraction (CON) [40, Definition 1.6] in 7 is if there 
exists e G (0,1] such that 

\\f( x ) ~ f{y)\\s < (! - e ) Ik - 2/lls , Vx, y G R n . 

2) Non-Expansive (NE) [50, Definition 4.1 (ii)] in 7 is if 

\\f(x) -f(y)\\ s < Ik-ylls- Var, y G R". 

3) Firmly Non-Expansive (FNE) [50, Definition 4.1 (i)] in 
Hs if for all x, y GP 

II f{x) ~ f(y) II 2 < Ik - yf s -\\f(x)-f{y)-(x-y)\\ 2 s . 

4) Strictly Pseudo-Contractive (SPC) [40, Remark 4, pp. 
13] in Hs if there exists p < 1 s.t. for all x,y G R" 

II fi x ) - f{y)f s < Ik - y\\ 2 s+p\\f(x)-f(y)-(x-y)\\ 2 s ■ 

a 

Contractiveness is a quite restrictive property, nonetheless 
this is the property commonly used in the MF control litera¬ 
ture, see for example [14], [9]. NE mappings are the simplest 
generalization of CON mappings. In the technical proofs we 
will make use of the following equivalent characterization of 
NE affine mappings. 

Lemma 3: Consider an affine mapping / : R" —► R”, x H>• 
f(x) := Fx + b, F G R” xra , b G R” and R G R” xn , R A 0. 
The following statements are equivalent: (1) / is NE in Hr. 
(2) \\F\\r < 1- (3) F t RF - R 4 0. 

Proof: (1) <G> || Fr — Fs||,r < ||r — s||_r Vr, s <G> 
\\F(r - s)|| fl < ||r - s\\ R Vr,s \\Fx\\ R < ||x||i* \/x 
(2) <^> H-Fzlllj < Iklll Vx <G> x T F T RFx < x T Rx Vx <G> 
x t (F t RF — R)x < 0 Vx <G> (3). ■ 

FNE mappings are a particular subclass of NE mappings, 
that includes for example the metric projection onto a closed 
convex set Proj c : R n —> C C R" [50, Proposition 4.8]. In the 
technical proofs we will make use of the following equivalent 
characterization of FNE mappings. 

Lemma 4 ([17, Lemma 5]): A mapping / : R" —> 
R" is FNE in H s if and only if \\f(x)-f(y)\\l < 

(x^y) T S(f(x)-f(y)), Vx, y G R”. ^ □ 

Finally, SPC mappings are a generalization of NE mappings. 
An equivalent characterization of SPC mappings can be given 
in terms of monotone mappings. 

Definition 5 (Monotonicity): A mapping / : R” —» R n is 

1) Strongly monotone (SMON) [50, Definition 22.1] in 
Hs if there exists e > 0 such that Vx,y G R", 
(f(x) - f(y)) T S(x-y)>e ||x - y |||. 

2) Monotone (MON) [50, Definition 20.1] in Hs if 

(f(x) — f(y)) T S (x — y) > 0, Vx,yG R". □ 

Lemma 5 ([17, Lemma 1 and 2]): 1) If / : R n — > R™ is 
MON and g : R n —t R n is SMON in Hs, then f + g is 
SMON in H s - 2) If Id - / is Lipschitz and SMON in H s , 
then / is SPC in Hs- □ 
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Appendix B 

Proof of Theorem 1 (Solution to Problem 1) 

Since x i *(z i ) = Proj (-(q i Q)~ 1 (Cz i + a)) in (12) 
is a continuous mapping in z l the mapping x*(z) in (13) 
is continuous in 2 . Consequently, the mapping A v (z) = 
P „x"(z) is continuous. By Assumption 1, the set P„X t x N 
is compact and convex. Therefore, by the Brouwer fixed 
point theorem [51, Theorem 4.1.5] the mapping Ay admits 
at least one fixed point. Consider now an arbitrary fixed 
point z = [z 1 ;...; z N ] G R^" of the aggregation mapping 
A v in (14), that is z l = YjL\ Pf x :i *(z j ). According to 
Definition 3, the set of strategies {jp := £**( 2 *)}^ is a 
NA Nash equilibrium if any agent i, given the strategies 
°f all the other agents, cannot improve its cost, that is 

x' = argmin yeX , P (y, Pjy + Y^i P ijX j ) • B Y definition 
of fixed point and using the fact that Pf = 0 Vi 

x l = a;**(z*) =argmin z*)=argmin J l {y-,Y^=\ P ii^) 

= arg min yeX i P(y, P^P +Y&i P ijX j ) 

= ^gmm yeX i P(y , Pjy + Y^i P ijX 3 )- ■ 

Proof of Lemma 1 (Properties of the extended aggregation 
mappings) 

The fact that A(z) = Xx*(z) admits at least one fixed 
point can be proven as done for A v (z) = VyX*(z) in 
the proof of Theorem 1. Let Dm := max xeXlxN ||a:||; 
then for any z G R Nn || A v (z) - «4.(z)| = \\VyX*(z) - 
lx*(z) || < \\P v - X\\\\x* (z)\\ < \\Vy - X\\D n . Hence 
su PzeR«" II Ay(z) - A(z)\\ < || Vy -X\\D n . By Assump¬ 
tion 4, Hindoo || P v - ^Ijvljvll = 0 which implies that 
lim^oo |j Vy — X\\ = lim^oo \\(P V - ^ 1^1^) (g)/„|| = 0, 
therefore lim^oo sup zeR ivn \\Ay{z) - A(z)\\ =0. ■ 


Proof of Lemma 2 (Fixed point sets) 

The proof of this lemma follows the lines of [52, Lemmas 
7.42, 7.44]. In particular, we start from the following lemma. 

Lemma 6 : Under Assumptions 1 and 4, given two se¬ 
quences (vh G N)^ =1 , (.Zh G such that 

lim/woo fh = 00 , z h = Ay h (z h ),\/h > 1 and Hindoo z h = 
z, then z = A(z). 

Proof: For the sake of contradiction, assume z / Aiz). 
Then there exists £ > 0 such that B(z,e) Cl B(A(z), e) = 0 , 
where B(z,e) := {x G R^" | ||x — z\\ < s). We know that 
the following statements hold: 1) Since, by Lemma 1, A v (■) 
converges uniformly to A(-), 3H\ > 0 : A Uh ( u) G 
B(A(u),s/2) V/i > iTi,Vu G R Nn ; 2) Since A(-) is 
continuous and Zh z, 3H 2 > Hi : A(zh) G 

B[A(z),e/2) Vft > H 2 ; 3) Since Zh —» z, 3H% > 
H 2 '■ Zh G B(z,e) V/i > H 3 . Therefore, from point 
1 ) with u = Zh and 2 ) we get that, for all h > H z , 
Zh = Ay h (z h ) G B(A(z h ), e/2) C B(A(z),e). Hence, 
using 3), we get Zh G (B(A{z),e)C\B(z,e)) 0, for all 

h > 7 / 3 , which is a contradiction. ■ 


We now prove Lemma 2. For the sake of contradiction, 
suppose that the claim is not true. Then there exists e > 0 
and a sequence (vh)h Li’ with Vh 00, such that Vh > 1 
y(Phi p ) > £• Equivalently, V/i > 1 3zh G T Vh such that 
y(z hl T) > e. Note that z h = A Vh (z h ) G Ty h Xi xN C 
(conv{A ’ i },^ =1 ) N =: X N for all h G N, and X N is compact 
by Assumption 1. Since any sequence defined in a compact 
set has a convergent subsequence, we can assume without 
loss of generality, that lim^oo Zh = z G X N . Note that 
y(zh,P) > e for all h > 1 implies that y(z,P) > e and 
hence z qL T. On the other hand, the sequences (vh)hLi 
and (z h )h = i satisfy the conditions in Lemma 6, therefore 
z = A(z) => z G T, which is a contradiction. ■ 


Proof of Theorem 2 (Solution to Problem 2) 

We follow a similar argument as in [17, Proof of The¬ 
orem 1], Given the fact that q: > q for all i, the map¬ 
pings x l *(z l ) = Proj q f?(—{qiQ)~ 1 (Cz l +Cj)) are uniformly 
Lipschitz with some constant L x > 0 , that is || x l *(z a ) — 
x l *{zb)\\ < L x \\z a — Zb\\ for all i and all z a , Zb G X. Moreover, 
given the fact that q, < q for all i, the quadratic cost functions 
./’ are uniformly Lipschitz in the compact set X x X with some 
constant Lj > 0, that is |«7*(a; a , z a ) — J l (xb, zf)\ < Lj(\\x a — 
Xb || + || z a — Zb||) for all i and for all x a ,Xb, z a , Zb G X. Let 
D = max ie ^ ||x||. For any (TV, v) G N 2 , consider an arbitrary 
fixed point z = [z 1 ;...; z N ~\ G R^" of the aggregation 
mapping Ay in (14), that is z l = Yrj=x Pfx' *(z j ) and 
define the set of strategies x 1 := Let x z * denote 

the optimal strategy, according to problem (7), for agent i 
given that the strategies of the others are fixed to {x :l } 

that is, P* := argmin^** J‘ (y, p y + Y^a jfX j ^j and let 
¥* := argmin^, J l (y, p x u + Y^n jf* 3 ) ■ Let us also 
define the associated costs J l * = J l (x l , + Yp^i W^ 3 ') > 
P* = P (x**, Xp* + Yf# jfX j ^ = ttim yeX i P(y, p y + 

Tt* 3 )’ ^d j- = P ^fXp' + Y^jjp) = 
mi n yeX i P (y, P P* + Y^i h* 3 ') ■ Note that p * < 

ji* < ji* ^ define P := jfP* + Y&i 7f% 3 ’ so that 
P = x l *(P), and note that 


\ji* _ ji* | < | ji* _ ji* | 

= I p p + Y&i ^ + Ef # < ^ j )\ 

<Lj(\\P~P\\ + p \\P-P^\\) 

= L J (\\P*(z i )-x i *(z i )\\ + ±\\x i -P*\\) 

< Lj(L x \\z l - 5*11 + ^11^ - ^*11) < Lj(L x \\P - ^11 + ^) 


and 11^ - F|| = - ±P* + Yrfi( p ij - £)^‘ll 

< - jfX* + p P - p Pf\ + Y^i \\( P ij - 7t^ j \\ 

< E^Li \\( P ij - 7f)x j \\ + jf\\P - ^*|| 

< D(YU \ P Z~w\ + 7?)< D (W pv - Tt^NllWoo + #). 
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Hence |P*- P*| < 2Lj(L x + 1)D{\\P V - + 

jf) := K(\\P U - ^IatI^Hoo + jj). Consequently, an ar¬ 
bitrary agent i can improve its cost at most by an amount 
£n, v ■= K(\\P V - ^Hjv 1 tv 11oo + ~k) if a11 other strategies 
{a:- 7 := x : ' *(P)} ... are fixed. Therefore, the set of strategies 

|a : 1 J is an £jv,i/-Nash equilibrium for the MF game in (7). 
K is a constant that does not depend on N, P or v and for 
any fixed N we have \\P V — -^ItvItvIIoo —> 0 as v —> oo. 
Consequently, for all £ > 0 and for any fixed N > N := If, 
there exists v such that for all v > D, we have En, v < e . ■ 

Proof of Corollary 1 

By the proof of Theorem 2 the set of strategies 
{P*(F)} 4= is an c/v^-Nash equilibrium for the MF game 
in (7), with e N ,„ = K(± + \\P V - £ljvl£||oo). By 
the properties of the matrix norm || P" — -^Ijv ijvlloo < 
VN\\P v - iljvl ^|| 2 = VN(r ma (P v - jflNlJ,) = 

|A| = VNiP n < \{N pP, where we 
used the fact that the matrix P" — is symmetric 

and, since P is symmetric, primitive and doubly stochastic, 
1 is a simple eigenvalue and it holds A (P v — = 

((A(P)n{l}) U {0}. Hence e N) „ < K(± + sfNpP). 

Proof of Theorem 3 (Convergence of Algorithm 1 and 2) 

To study the convergence of Algorithm 1 and 2, we start by 
studying the regularity properties of the mapping A.,,. First, we 
analyze the regularity properties of the constrained minimizer 
x l * of problem ( 12 ). 

Lemma 7 (Regularity of the constrained minimizer): For 
any i £ Z[l, N], let x l *(z l ) be as in (12) and Af as in (21). 
The following facts hold: 1) If A/, >- 0, then the mapping 

x l * is a CON in TLq ; 2) If M* y 0, then the mapping x l * is 
NE in TLq; 3) If —qtQ 4 C 4 0, then the mapping x l * is 
FNE in TL-c ; 4) If 0 A C, then the mapping —x l * is MON 
in Lie- □ 

Proof: The proof is an extension to the proof of [17, The¬ 
orem 2 ] for the case of heterogeneous cost functions and fol¬ 
lows from the proof of [21, Theorem 2], It is reported here for 
completeness. Note that x l *(z l ) := Proj^P(— (qiQ) -1 (C z l + 
Ci )). 1) and 2) follow from [17, Theorem 2] by setting 
Q = QiQi A = 0 and by noting that x l * is CON/NE in TLq if 
and only if the same holds in TL qi Q . 3) From [50, Proposition 
4.8] we have that Proj^,, is FNE in Hq, hence by Lemma 4 
in Appendix A we have that, for all v, w £ R", 

( Pr °j xi(~( , pQ')~ 1 ( Cv +Ci)) - Pr °j %{~(qiQ)~ 1 (Cw+ c i))) T 

■ Q (“( q l Qy 1 C(v - w)) > 

Vm}® i (-(q i Q)~ 1 (Cv+c i ))-'Pro}Q i (-(q i Q)~ 1 (Cw+c i )) 

Q 

^ (x i *{v)-x i *{w)) T (-qr 1 C)(v - w)> 11^*(v) — *(w)||^ 

<^=> (x**(v) — x l *(w)) T (—C)(v — w) > ||P*(u) — x l * (if) 11 ^ q 

(24) 

Therefore, if qiQ y —C y 0 then from (24) we get that 

(x l *(v) — x**(ui)) T (— C)(v — w) > ||x**(v) — 


which implies by Lemma 4 that x l * is FNE in 
TL-c- 4) From the last inequality in (24), we get that 
(— x l *(v) + x l *(w)) T C(v — w) > 0 for all v,w £ R". If 
C A 0, this implies that — x % *(-) is MON in He- ■ 

Next, we show that the regularity properties of the con¬ 
strained minimizer x 1 * in ( 12 ) are inherited by the extended 
mapping x* in (13). 

Lemma 8 (Regularity of the extended optimizer): 

Let x*(-) be the extended mapping defined in (13). 
Given Mi as in (21), the following facts hold. 1) 
If Mi y 0 Vi £ Z[l, N ], then the mapping x * is a CON in 
TLi n ®q\ 2) If Mj y 0 \/i £ Z[l,Af], then the mapping x * 
is NE in H In ®q-, 3) If -qiQ 4 C A 0 Vi G Z[1,N), then 
the mapping x* is FNE in TLi N ®r-c)', 4) If 0 A C, then the 
mapping -x*(-) is MON in TLi n ®c- □ 

Proof: 

1) For all i £ Z[l,iV], if Af y 0 then, by 

Lemma 7, the mapping x l * is a CON in Hq, 
with some rate Si £ (0,1]. Therefore, for any 
r,s £ R Nn we have ||a;*(r) - **(s)||f iV( g,Q = 

|| [x 1 *(r 1 )-x 1 *(s 1 );...;x N *(r N )-x N *(s N )] ||| v0Q 

= ||a; 1 *(r 1 ) — a: 1 *(s 1 )||q + ... + ||x Ar *(r Af ) — 

^*(^)|| 2 Q < (1 — <5 1 ) 2 ||t ’ 1 — S 1 IIQ + ■ • ■ + (1 — 
<5Tv) 2 ||r JV - S JV ||^ < (l-^n iGn q N ]S t nr-sr iN0Q . 
Note that S := min^^M jv] S, is strictly positive since 
N is finite. 

2) If Mi y 0 then, by Lemma 7, the mapping x l * is NE 
in TLq. The proof is the same as in the previous point, 
with Si = 0 for all i. 

3) If — qiQ C -< OVi £ Z[l, TV] then, by Lemma 7, 

the mappings x 1 * are FNE in TLr^c)- Therefore, by 
Lemma 4, for all r,s £ R Nn we have ||a:*(r) — 

**( s )HL®(-c) = eZi w ^) - 

< - s*) T (—C)(x**(r i ) - 

= (r- s) t (I n 0 (-C))(x*(r) - x*(s)). 

4) If 0 A C then, by Lemma 7, the mappings — x l * 
are MON in TLq■ Therefore, for any r,s £ R ;Vn 
we have (— x*(r ) + x*(s)) T (In 0 C) (r — s) = 
Eti (-^*(r l ) + »**(«<)) f C ( r l - s l ) > 0. 

■ 

The following lemma proves that the linear mapping x i—>• 
V v x is NE, if ||P|| <1. 

Lemma 9: Consider P £ R NxN , v £ N and P u = P v 0l n . 
For any S £ R nxn , S y 0, let S := I N 0 S. If ||P|| < 1, 
then HP^IIs < 1 . □ 

Proof: The condition ||P|| < 1 implies ||P^|| < ||P|| I/ < 
1. By Lemma 3 (R = In) this implies ^P" T P v — In'' j =4 0. 
Moreover, by Lemma 3 (R = S), ||P„||,s < 1 
V„ T SV v -S 4 0 ^ (P v 0 Inf (In 0 S ) (P v 0 /„) - 
I N 0S 4 0 ^ (P" T ® (I N 0 S) (P v 0 I n )-I N 0S 4 

0 (p vT I N P V 0 IISin) - In 0 S 4 0 (p vT P v ) 0 
S-I N 0S 4 (P" T P 1/ -I n )0S4O. Since 5^0 

and ^P" T P v — In) 4 0, by the properties of the Kronecker 
product we conclude that (p uT p u — I N ) 0 S 4 0. Finally, 
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by the previous equivalence, we have ||'P ! ,||.s < 1- ■ 

Note that a single iteration of Algorithm 2 updates the signal 


z ik ) = 


'{kV 


Hk) 


according to the mapping 


A„ uV2 {z) := Vv 2 x*(V Ul z) 


(25) 


and Ai.„ = A„. The following proposition characterizes the 
regularity properties of Ay lt y., for different choices of i'\, i> 2 . 


Proposition 1 (Regularity of the update mapping): The 
following statements hold. 

1) If Mi >- 0 Vi £ Z[l, N] and ||P|| < 1, then the mapping 
Aom = A ly in (14) is a CON in Ui n ®q\ 

2) If Mi 0 Vi G Z[l, N] and ||P|| < 1, then the mapping 
Aq, u = Ay in (14) is NE in P/ n ®q; 

3) If -qiQ =<: C ■< 0 Mi £ Z[l, N], a £ 2N and P = P T , 
then the mapping A" ^ in (25) is FNE in P 7jv0( _ c) ; 

4) If 0 -< C, v G 2N and P = P T , then the mapping 

Av,» in (25) is SPC in P/ N 0 c- □ 

Proof: 

1) Let S := In ® Q • By Lemma 8 , x* is a CON in 77s 
and, by Lemmas 3 and 9, V v is NE in 77, s. Hence the 
mapping Ay = V yX*, composition of a CON mapping 
and a NE one, is a CON in 77s- 

2) Analogous to the proof of statement 1, with x* NE. 

3) Let S := C). From Lemma 8 , x* is FNE in 77s- 

Since P is by construction a row-stochastic matrix and 
P = P T , P is a doubly stochastic matrix. Consequently 
||P||i = ||P||oo = 1- From Holder’s inequality, we have 
||P|| < i^/HP|| i • ||P||oo = 1. Therefore, from Lemma 9, 
\\Vz\\ s < 1. Moreover, P = P T implies V z = V z T 
and P-5 = (P* ®I n )(I N $S) = (P* ®S) = SV». 

Therefore for any r, s £ E JVn , 

||^ | , f (r)-^ | , | (s)||| (26) 

= \\V,(x*(V,(r)))-T,(x*(T,(s)))\\ 2 s 

< Infill \\x*{V,{r))-x*{V,{s))\\l 
<||aC(P | (r))-aC(P | (s))||| 

< (V,r~V,s) T S(x*(V~(r))-x*(T%(s))) 

= (r - s) T Vr T S K(P f (r)) - x*(V,(s))) 

= (r-s) T SV | (x*(V%(r))-x*{V%(s))) 

= (r- s) T S(Ar i% {r) - A%» (a)), 

where the third inequality derives from x* being FNE. 
The inequality in (26) implies that Ak^(-) is FNE in 
P 5 , by Lemma 4. 

4) Let S := In < 8 > C. From Lemma 8 , -x*(-) is MON in 
PLs- Moreover P = P T implies that V e 1 S = SV |. 
Therefore for any r, s £ R iVn , 

= (-V, (x*(V$ (r))) + V,(x*(V,(s)))) T S (r - a) 
= (-x*(V*(r)) + x*(V,(s))) T V, T S (r - a) 

= (-®*(P*(r)) + : r*(P | ( S ))) T 5 (P 5 r - P*s) 

= (—x*(r) + tc*(s)) T S (r — s) > 0, 


which implies that — Az^(-) is MON in 77s- Since 
7jVn( - ) is SMON, it follows from Lemma 5 that /jv n ( - ) - 
I (•) is SMON and Lipschitz, since it is the compo¬ 
sition of Lipschitz mappings. Consequently, Ay : ef) is 
SPC in Ps, by Lemma 5. 

■ 

We are now ready to prove Theorem 3. We start by noting 
that, since the sets X 1 are convex and compact for every i £ 
Z[1,N], the mapping Ay, M2 : Vy 2 X lxN -> Vy 2 X lxN is 
defined from a compact and convex set to itself. For simplicity 
let a (fe) (vi ,v 2 ,z 0 ) := [Al lfi {z {k) )\ ...; A^ lfi {z {k) )\. 

Proof of statement 1 and 2 of Theorem 3 : From Proposi¬ 
tion 1 , under the assumption of statement 1 (or statement 2 ), 
«4o,„(-) = Ay(-) is CON (or NE) in 77 / w ®q. Therefore, 
by using 4> p B (or 4> K ) Z( k ) converges to a fixed point of 
the mapping Ay [40, Theorem 2.1] ([40, Theorem 3.2]). 
The proof is immediate upon noticing that, since = 0, 
a {k) := 'Pv 1 z (k) — z (k)- Moreover, a CON mapping has a 
unique fixed point [40, Theorem 2.1], 

Proof of statement 3 and 4 of Theorem 3 : From Proposi¬ 
tion 1, under the assumption of statement 3 (or statement 4) 
the mapping Z( k ) K > Ar^(z( k )) is FNE (or SPC). Therefore, 
by using <t> p B (or <hj pl ) z lk) converges to a fixed point z of 
the mapping A% t %, [53, Section 1, p. 522] ([40, Fact 4.9, p. 
112]) . Note that := Vy 1 z^ hence, for = |, 
converges to a := Vgz, which is a fixed point of the mapping 
Ay since z = A%,%(z) => z = V%x*(V%(z)) => V%z = 
VyX*(Vz(z)) => a = VyX*{a) => a = Ay(a). ■ 


Proof of Corollary 2 (Convergence to a NA Nash equilibrium 
in the opinion dynamics application) 


For every agent i 
holds Mi = 


q,.Q -C 


-C qiQ 



Z [ 1 , iV], the following equivalence 

(1 + 0i) 1 

1 (1 + Of) 

hence the eigenvalues of M t are 0i and 1 + 0i, both with 

multiplicity n. It follows that if 0i > 0 then Mj >- 0, if 

0i > 0 then Mj R 0. Consequently, by Theorem 3 the given 
conditions guarantee convergence of Algorithm 2 to a fixed 
point z of the aggregation mapping A\. Since Assumption 2 

is satisfied. Theorem 1 guarantees that the set of strategies 
1 N 


! is a NA Nash equilibrium. 


Proof of Corollary 3 (Convergence to an e-Nash equilibrium 
in demand-response methods) 


Given the cost function in (3) with p(af) := 
the following equivalence holds Mi = OxQ 

—o 


Act t + Po, 
-67 ' 

QiQ 


Pi 2 
Pi 


® It , hence the eigenvalues of Mj are p t + x 2 


and pi — A j both with multiplicity T. It follows that if 
Pi — A > 0, then Mj >- 0 while if p-i — A > 0 then Mi R 0. 
Moreover, C = | J T >- 0 . Consequently, by Theorem 3 the 
given conditions guarantee convergence of Algorithm 2 to a 
fixed point z of the aggregation mapping A„. The conclusion 
follows from Theorem 2. ■ 
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Hierarchical control 

Lemma 10: If Pm £ R MxM satisfies Assumption 4, then 
also P = Pm < 8 > glsls does. Moreover, if ||Pm|| < 1, then 

|/ J II< 1- 

Proof: By the properties of the Kronecher 

product we have lim,,-^ (Pm < 8 > g 1 b 1 b) = 

lim^-^oo {Pjf ® -glslj) = (ifI jwIm) ® (glslj) = 
Wb^mbImb = ~k^ N ^iv- Moreover, ||P|| = 

y ^prp) = = 

\JP i P M P M ® 5 1-Bljg) = P ( P M P m) = ||-Pm|| < 1) 
where we used the fact that the matrix -g -1 sis has all 
the eigenvalues in 0 except one which is equal to 1 , all 
the eigenvalues of Pm Pm are nonnegative real and the 
eigenvalues of the Kronecker product are the product of the 
eigenvalues of the two matrices. ■ 

Appendix C 

From the proof of Theorem 2 for any e > 0 and N > 
N := the set of strategies {x * 1 * (z 1 )}^ is MF £-Nas h 
equilibrium if \\P V - < £ d ■= ~ TP If t he 


Algorithm 3: Precomputing v on a given network P 

Initialization: k 0, fix Ed > 0, s l 0 Vi, P° = I n . 
Iterate until s' 1 — 1 : Each agent i synchronously 

1 ) receives the weight coefficients {P^jjjLi from each 
of its neighbors h £ Af l 

2) computes {P* +1 = p ih P hj} jLi 

3) if Ef=i l ^ +1 \< £ d, then P <- 1 , else <- 0 

4) repeats N times s l t— min 3 ej ^.. u r i }{s J } 
fc <- k + 1 

Output: v <— k. 


network is strongly connected and Assumption 4 holds, then 
it is possible to compute, for any Ed > 0 and in a distributed 
fashion, a value of v such that || P" — -T IjvliHoo < £ d by 
using Algorithm 3. The proof is based on the following fact: 
suppose that each agent i has a value of ,P £ {0,1}. Since the 
graph is strongly connected after at most N iterations of the 
update s l <r- min je ^ !U {,} {.s J } the agents reach consensus to 
a value s which is 1 if all the s* at the beginning were equal 
to 1 and is 0 otherwise. Therefore if Algorithm 3 terminates 
it must be at a value v for which EyLi \ P ij ~ xrl < £ d for all 

i, which in turn implies \\P U — -^Ijvijvlloo < £ d- Note that 
Assumption 4 guarantees that for each agent i, there exists 
a value v l such that E /Li \ p tj — ^1 < £ d for all k > 
Hence for k > max{F*}"_ 1 , s' = 1 for all i and Algorithm 3 
is guaranteed to terminate in a finite number of iterations. 
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